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Abstract 



Top quark production offers the unique opportunity to search for a charged 
Higgs boson (H^), as the contribution from t — > H + b — ► r + vb can be large 
in extensions of the Standard Model. We use results from a search for top 
quark pair production by the Collider Detector at Fermilab (CDF) in the er + 
Fj T +jets and [it + Fj r +jets signatures to set an upper limit on the branching 
ratio of B(t — > H + b) in 106 pb _1 of data. The upper limit is in the range 0.5 

5 



to 0.6 at 95% C.L. for H + masses in the range 60 to 160 GeV, assuming the 

branching ratio for H + — > tv is 100% . The r lepton is detected through its 

1-prong and 3-prong hadronic decays. 
PACS numbers: 12.60.Fr, 14.80.Cp 
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Many extensions of the Standard Model (SM) include a Higgs sector with two Higgs 
doublets, resulting in the existence of charged (H ± ) as well as neutral (h, H°, A) Higgs 
bosons. The simplest extensions are the Two-Higgs Doublet Models (2HDMs) fll], in which 
the extension consists only of the extra doublet. In a Type I 2HDM only one of the Higgs 
doublets couples to fermions, while in a Type II model one Higgs doublet couples to the 
"up" fermions (e.g., u,c,t), while the other couples to the "down" fermions (e.g., d,s,b). The 
Minimal Supersymmetric Model (MSSM) || is a further extension of the SM, and has a 
Higgs sector like that of a Type II 2HDM. 

If the charged Higgs boson is lighter than the top quark 0-0], i.e. m#± < {m top — m&), 
the decay mode t — > H + b will compete with the SM decay t — > W + b. The consequence is that 
tt production and decay will provide a source of Higgs bosons in the channels W^H^bb and 
H + H~bb produced with a strong-interaction rather than the weak-interaction cross-section 
of direct H + H pair production. In addition, the signature from top pair production and 
decay is much cleaner than that of the direct production with respect to QCD background. 

In a 2HDM and in the MSSM the branching ratio for t — > H + b, £>y b , depends on the 
charged Higgs mass and tan/3, the ratio of the vacuum expectation values for the two 
Higgs doublets. Figure |T] shows the expected branching ratio from a leading-log QCD 
calculation || in the MSSM for three different charged Higgs masses m#± = 60, 100, 140 
GeV/c 2 as a function of tan/3. For tan/3 ^ 1 and tan/3^ 70 the MSSM predicts that the 
decay mode t — > H + b dominates. Also shown in Figure p] is the predicted branching ratio 
in the MSSM at lowest order for the decay of the charged Higgs boson into a charged r— 
lepton and a r-neutrino (B^J), which has little dependence on the charged Higgs mass. For 
tan/3 > 1 the decay H + — > r + v T is predicted to dominate over the other main decay mode, 
H + — ► cs, and for tan/3 > 5 the branching ratio B^ v is expected to be nearly 100%. Thus, 
this model would predict an excess of top events with tau leptons over the number expected 
from SM events in which tt —>■ W + W~bb, followed by W — » tv. 

Recent calculations, however, have shown that at large values of tan/3 the predicted 
branching ratio for t — > H + b is highly sensitive to higher-order radiative corrections, which 
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are model-dependent |7j. Limits in the tan/3 — m H ± parameter plane consequently depend 
critically on the parameters of the model. However the direct search for the signature of a 
r lepton in top decays allows us to set an upper limit on the branching ratio of t — > H + b, 
assuming the branching ratio for H + — ► tv is 100%, for example. 

Previous searches for the charged Higgs boson in top decay have been in the r + E t chan- 
nel g| [§, ££+# T +X (£ = e or //, X=anything) channel the F^ + r+jets channel 



the £ +jets channel [ 13t,p~4 1 , and the E t + t5 + 0+ jet (O = e, /i, r or jet) channel |L5|. Both 
Ref. fill] and Ref. |15[ select events with a Fj T trigger, while Ref. is an indirect search 

using a disappearance method. Searches for direct production at LEP set a lower limit on 
the mass of 69 GeV/c 2 flRj. Indirect limits have also been set from measurements of the 



rate for the decay b — > s^y |L7). However higher-order calculations have shown that in both 
2HDM models |1| and the MSSM |19[ these limits are also highly model-dependent. 

The CDF collaboration has published a search for r leptons from decays of top quark 
pairs in the It+% t +2 jets+X (£ = e, /i) channel pD| , where events were selected by requiring 
the presence of a high-p^ e or /i. We present here the constraints that this analysis (the"£r" 
analysis) imposes on the branching ratio of the top quark into a charged Higgs boson. This 
was suggested in Ref. [21], where the authors compare the CDF data with a generator-level 
Monte Carlo calculation for the number of expected events from charged Higgs decay. 

In this paper we start with the number of top candidate events found in the It + % T + 2 
jets+X data in the analysis of Ref. |2(J . We then apply the same selection criteria to Monte 
Carlo events that contain top quark pairs in which one or both top quarks decay to the 
charged Higgs (i.e. ti W ± H T bb and ti — > H + H~bb), for different Higgs masses. We 
assume there are no top quark decays other than t — > W + b and t — > H + b. We perform 
a full calculation of the acceptances including detector effects, and determine the expected 
number of events due to Higgs production and subsequent decay. From this we can set a 
limit on the branching ratio t — > H + b. 

The selection used in this analysis requires high-p^ inclusive lepton events that contain an 
electron with Et > 20 GeV or a muon with pt > 20 GeV/c in the central region (|?7| < 1.0). 
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The other lepton must be a tau lepton, also in the central region, with momentum pt > 15 
GeV/c |25[]. Dilepton events from ti decays are expected to contain two jets from b decays 
and large missing transverse energy from the neutrinos. Therefore, we select events with 
> 2 jets (with Et > 10 GeV and \r]\ < 2.0), and with large Fj t significance (S^ > 3), as 



described in detail in Ref. 



Two complementary techniques, one which identified the r lepton starting with clusters 
in the calorimeter, and another which started with a high px single track, were used for 



identifying hadronically decaying r's [p0 |. Here, we combine the two tau selections by 
accepting events which pass either set of criteria. Both techniques find the same four top 
dilepton candidates in 106 pb _1 of data. The total acceptance of the combined selection 
for SM top quark pairs decays, i.e. the events that pass the final cuts divided by the 
number of generated ti events, is (0.172±0.014)%. We expect a total of 3.1±0.5 events 
from background sources. The dominant background is due to Z/7 -> T + r~+jets events 
(1.8 ± 0.5 events), and to W+ > 3 jets events where one jet is misidentified as a r lepton 
(1.0 ±0.1 events). We expect 0.3 ±0.1 background events from WW and WZ production. 
We calculate the number of expected events in the It channel by combining the ti cross 
section, the luminosity and the total acceptance. For the ti cross section we use the CDF 
measurement in the "lepton±jets" channel, where one W decays leptonically and the other 
W decays hadronically. This yields the most precise determination of the ti cross section in 
a single channel, a t j = 5.1±1.5 pb pTfl . Using this cross section we expect 0.9 ± 0.1 events 



from SM tt decay in the er and fir channels. 

Although the identification of b quarks was not part of the search criteria, three of the 



four candidate events contain at least one 6-tagged jet p3fl , while we expect 0.2 tagged 
events from SM non-tt background pO| . I* 1 the following we will use the combined tau 
selection for our results. 

If a charged Higgs boson is present all three of the final states W + W~bb, W^H^bb, and 
H + H~bb can contribute to the It channel. The total acceptance for top decay in the It 
channel is given by 
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Air _ /-i _ »t \2 aIt i 

2(1 - B^AB\^ V A% U + 

(^ b ) 2 (0 2 ^ • (1) 

Here A^ w is the total acceptance of the event selection criteria for the case where the 
ti pair decays into W + W~bb. It includes the geometric and kinematic acceptances, the 
efficiencies for the trigger, lepton identification, and cuts on the event topology, and all 
branching ratios of both the r and the W boson |24" |. Similarly, Ay VH and A^ H are the 



respective total acceptances for the ti pair decays into W ± H Zf bb and H + H~bb, but where 
the branching ratio of the top to Higgs (B^) and of the Higgs to tau (Bf v ) have been 
factored out explicitly We assume that B^ v is 100%, as it would be at large tan/3 in the 
MSSM, and set a limit on B^. 

We use a top quark mass of 175 GeV/c 2 . Monte Carlo simulations of ti production 
and decay in the three modes W + W~bb, W ± H T bb, and H + H~bb provide estimates of the 
geometric and kinematic acceptance, A geom .p T , and of the efficiency of the cuts on the event 
topology for different Higgs masses (m H ± = 60, 80, 100, 120, 140, 160 GeV/c 2 ). We use the 
pythia p2] Monte Carlo to generate ti events, the tauola package |[26|| , which correctly 



treats the r polarization, to decay the tau lepton, and a detector simulation. The selection 



of events is identical to that described in detail in Ref . [ 20| . The efficiencies for electron and 
muon identification are measured from Z° — > e + e~ and Z° — > fi + fi~ data. 

Figure shows A geom .p T , the efficiency €j et of the 2-jet cut, the efficiency e# T of the cut 
on the total transverse energy P0fl , and the efficiency of the cut on the fir significance, 
as a function of Higgs mass. As m#± increases the tau leptons become more energetic and 
Ag e0 m-p T increases. When m#± approaches m t0 p the b jets instead become less energetic and 
€j et drops rapidly. Figure |3| shows the resulting values for A^ H and A e ^ H versus m H ± ; the 
numerical values are listed in Table |. Note that relative to Ahh, A^h has a factor of 2/9 
in it due to the branching ratio for W — > iv, while thhe factor of two due to the two possible 
charge combinations W + H~ and W~H + is explicitly included in eq. (1). Overall, the total 
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acceptance (A^ t ) is rather insensitive to the value of the Higgs mass, ranging between 0.7 % 
and 1.3 % until m H ± approaches m top . This is to be compared to the acceptance A\^ w in 
the W+W- final state of 0.17%. 

The expected number of events in the £t channel is given by 

NZ p = t r a >£>A£ t (E% Sb ,m H ±) (2) 

and depends on B^ h , the Higgs boson mass, and <r t j, the total top pair production cross 
section. Rather than use the theoretical prediction for a t j , for each value of £>^ b we normalize 
to the observed number of events in the 'lepton + jets' channel with a secondary vertex tag, 
taking into account the contributions from the three separate decay final states of W + W~bb, 
W ± H T bb, and H + H~bb, calculated using the full Monte Carlo simulation and the updated 
tagging efficiency p7j . We have checked that the calculation gives the value of a t j = 5.1 pb 
in the SM case of B\± h = 0, in agreement with the CDF standard model analysis of the top 
cross section |p7[| , as it must be. We thus calculate a t j from the number of observed events in 
the lepton plus jets channel with a secondary vertex tag, N l+ i ets = 29, the expected number 
of SM background events, B i+ i ets = 8.0 ±1.0, and a total acceptance A t ^ ets (B^, m H ±) that 
takes the W ± H T bb and H + H~bb decay modes into account. This can be written as 

j\j~£+jets j^l+jets 



ft C-A l +r tS {B^m H± ) 
where A l t ^ ets is given analogously to A^ ot by 



(3) 



Ai+jets _ jot \2 A e +3 ets i 

^tot — \ L &Hb) ^WW 

2(1 - B^B^A^ + 

(^Hb) 2 < + ^ • (4) 

Figure |] shows how a t j increases as B^ becomes larger. The contribution from H + — > cs 
decays is neglected, as we have assumed B^ u = 1. For a large branching ratio into H + b, 
the H + H~bb mode becomes dominant and the leptons (e or /i), which in this case originate 
from tau decays, have a softer p T spectrum than leptons produced in W decays, and A l ^ ets 
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decreases. Figure |5| shows the expected number of events versus B^ from each of the 
W + W~bb, W ± H^bb, and H+H-bb decay modes for m H ± = 100 GeV/c 2 . 

Based on the observation of 4 events and the predicted background of 3.1 ± 0.5 events, 
we calculate a 95% CL. upper limit on Higgs production of 8.1 events. When calculating 
the limit, we include the systematic uncertainties, which are dominated by uncertainties on 
j\ji+jets (26%), tau identification (11%), b tagging efficiency (10%) and Monte Carlo statistics 
(8%). Then, to determine a limit on the branching ratio £>Ht>> we calculate the number of 
events expected versus B^ h for different Higgs masses in steps of 20 GeV/c 2 . Figure || shows 
the region excluded at 95% CL. as a function of the branching ratio of t — > H + b. The upper 
limit is in the range 0.5 to 0.6 at 95% CL. for H + masses in the range 60 to 160 GeV. 

For the special case of the MSSM, although the branching ratios have been shown to be 
strongly model-dependent, for the Higgs mass parameter \i < the SUSY QCD and QCD 
corrections come close to cancelling, and the next-to-leading order prediction is almost 
unchanged from the tree-level result J7|. Figure [7| shows the expected number of It events 
versus tan/3 from each of the W + W~bb, W ± H T bb, and H + H~bb decay modes for m H ± = 
100 GeV/c 2 , at lowest order in the MSSM. The shapes of the curves are mainly due to the 
variation of the branching ratio B^ as a function of tan/3. Figure |^ shows the excluded 
region in the plane of and tan /3, again at lowest order in the MSSM. In the region at 
large values of tan f3 the tbH + Yukawa coupling may become non-perturbative (see Ref. @). 
In this case the limit is not valid. 

We compare our results to those of Ref. pi] . We find that the acceptance is smaller 



by about a factor of two. The limits presented in this letter use the correct W + W bb, 
W ± H^ f bb, and H + H~bb acceptances, including the correlations among the different objects 



(e, /i, r, 6-quark) in the events. The insight of Ref |2l] that this will be a channel of much 
interest in Fermilab Run II remains intact, however. 

In conclusion, we have used the data from the CDF search [^(J for top quark decays into 
final states containing a light lepton (e or /i) and a r lepton, detected through its 1-prong 
and 3-prong hadronic decays, to set a limit on the branching ratio of the top quark into the 
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charged Higgs plus a b quark, B^. The limit ranges from 0.5 to 0.6 at 95% C.L. for H + 
masses in the range 60 to 160 GeV, assuming the branching ratio for H + — > rv is 100%. 
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TABLES 



Mm 99 s A* H (%) A% H (%) 

60 0.91±0.06 1.00±0.06 

80 0.98±0.06 1.17±0.06 

100 1.11±0.06 1.32±0.07 

120 1.08±0.06 1.32±0.07 

140 0.67±0.05 0.98±0.06 

160 0.72±0.05 0.32±0.03 



TABLE I. The total acceptance versus the mass of the charged Higgs boson for the £t + ~$ t + 2 
jets +X analysis. The uncertainties are statistical only. These numbers are to be compared to the 
acceptance for SM top quark pair decays of A l ^ w = (0.172 ± 0.014)%. The larger acceptance with 
the charged Higgs is primarily due to the larger branching fractions into r leptons. 



17 



FIGURES 
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m Hi gg s IGeV/ c 2 ] rn Hig9S [GeV/c 2 ] 
FIG. 2. Contributions to the total acceptance in the £t + $ + 2 jets+X channel versus the mass 

of the charged Higgs boson. The circles are for the W^H^bb decay of the ti pair; the squares for 

for the H + H~bb decay. 
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FIG. 3. Total acceptance in the "tau dilepton" channel, versus the mass of the Higgs boson. 
The circles are for the W^H^bb decay of the tt pair; the squares for the H + H~bb decay. 




19 



18 




BR(t->Hb) 

FIG. 5. The predicted number of events for 106 pb~* of data versus the branching ratio for top 
decay into H + b for mHi ggs =100 GeV/c 2 . The graph shows the contributions from the W + W~bb, 
W^H^bb, and H + H~bb channels separately. 
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FIG. 6. The region excluded at 95% C.L. for charged Higgs production versus the branching 
ratio for top decay into H + b. 
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tan p 

FIG. 7. The predicted number of events at lowest order in the MSSM for 106 pb _1 of data 
versus tan/3, for mHiggs=W0 GeV/c 2 . The graph shows the different contributions from the 
H + H~bb and W^H^bb channels separately. 
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FIG. 8. Excluded regions (95% C.L.) at different values of tan/3 for charged Higgs production, 
at lowest order in the MSSM. The coupling tbH + may become non-perturbative in the region at 
large values of tan /3, and the limit does not apply. 
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